Method and device for evaluating view images

ABSTRACT

The invention relates to a method for evaluating at least one view image (M) in order to image at least one viewpoint, which is provided in relation to at least one scene shot (S) of a scene (12), and/or at least one view direction of at least one person (10), said view direction being provided in relation to the at least one scene shot (S), towards a reference, the content of which matches at least one part of the scene (12). The at least one scene shot (S) is provided with the at least one assigned viewpoint (B) and/or the at least one assigned view direction, and the reference (R) is provided together with a result (B′) of the at least one view image (M). Furthermore, the at least one view image (M) is evaluated by means of at least one predefined quality measurement (GM), and the result of the evaluation is provided.

The invention relates to a method for evaluating at least one view imagein order to image at least one viewpoint, which is provided in relationto at least one scene shot of a scene, and/or at least one viewdirection of at least one person, said view direction being provided inrelation to the at least one scene shot, onto a reference, the contentof which matches at least one part of the scene. At least one scene shotis provided with the at least one assigned viewpoint and/or the at leastone assigned view direction, and the reference is provided together witha result of the at least one view image. The invention also includes adevice for evaluating at least one view image.

View images, which are also called view direction images or viewpointimages, find application in many areas. One such area of application is,for example, consumer research or even the conducting of studies. Forthis purpose, eye trackers, e.g., head-mounted eye trackers, can be usedwhich determine the view directions of a person while this person looksat a particular scene, e.g., his real surroundings or a display on ascreen. In addition, a scene camera, which at the same time takespictures of the surroundings of the person, covering at least a largepart of the current field of view of such a person, can be attached tosuch a head-mounted device. As a result, for a specific point in time,the view data determined at that time by the eye tracker can becorrelated with such a scene shot recorded at that time, and thedirection in which the person looked at that time in relation to hissurroundings can be determined therefrom. A procedure of this kind canthen be carried out for an entire scene video so that each individualscene shot of this scene video then contains a corresponding viewpointor a view direction of the person. In this way, it can be determined inwhich direction, for how long or how often a person looked at specificobjects in his environment during this experiment or this study. On thebasis of such test results, it can be determined, for example, whichproducts on a supermarket shelf attract more attention and which less;which advertising catches the eyes of individuals more and which less,and so on. Normally such experiments are carried out not only with asingle person but with many different persons, e.g., different ageclasses, genders, and so on in order to obtain statistically meaningfulresults from the findings.

However, since an evaluation of every single scene shot in such numerousscene videos with the corresponding view data is a very laboriousmatter, there is the possibility of also carrying out such an analysisautomatically or at least simplifying it. This is where view images areused. It is thereby made possible, for example, to represent all of theviewpoints contained in a scene video on a single common reference. If,for example, a person looks from different perspectives at a particularshelf holding different products during the experiment, a picture ofthis shelf can, for example, be provided as a reference. All viewpointsrecorded in relation to the scene video of this shelf can then be imagedonto this single reference shot with the shelf. As a result, a muchbetter impression can be gained as to which objects on the shelf areviewed more or less frequently, and the comparability of the view datacan be made significantly better. In this case, such a view image caneven be created covering a plurality of individuals so that the viewingresults of all individuals are imaged onto a common reference. Inaddition to this application example, there are also however numerousother possible applications for view images which image viewpointsand/or view directions in relation to a scene shot onto a reference.

On the one hand, such view images can be created manually, e.g., by oneperson viewing each scene shot together with the viewpoint it contains,and then plotting the corresponding viewpoint on the correspondinglocation, i.e., on the corresponding object or in the corresponding areaof an object on the reference image. Since creating such a view imagemanually is very time-consuming and thus also cost-intensive, especiallyin the case of numerous scene shots and view data, there is also thepossibility of creating such view images automatically. Certainalgorithms can be used for creating such view images automatically. Onesuch algorithm references, for example, the scene shot to the referenceshot, and determines therefrom, for example, a transformation whichimages the scene shot, which was, for example, taken of a scene from acertain perspective, onto the reference, which was also taken of thisscene but from a different viewing angle. Next, the transformation soobtained is applied to the viewpoint determined in relation to the sceneshot, which yields the corresponding imaged viewpoint on the referencein relation to the reference. Such view images can however be obtaineddifferently depending on the application.

In either case, i.e., both with the manual view image and with anautomatic view image, errors can however occur, e.g., a viewpoint fromone scene shot being imaged onto a wrong area or wrong object of thereference, or a viewpoint in a scene shot not being imaged at all ontothe reference because the corresponding object was erroneously not foundthere. In the case of manual view images, errors of this kind areusually due to carelessness, fatigue, or the like. With automatic viewimages, this is largely due to the algorithms responsible for the viewimage, whereby errors in the referencing between images or in objectrecognition or classification can occur. This means that the results ofsuch view image, that is, the viewpoints eventually imaged onto thereference and, for example, statistical statements derived therefrom orthe results of other further processing steps, can be more or lessreliable, wherein it is precisely the user who is left more or less inthe dark in this respect. In order to nevertheless be able to obtainmeaningful and reliable results, the quantity of input data must beincreased accordingly. In other words, more scene videos or longer scenevideos and more view data must be recorded and evaluated, which in turngreatly increases the effort spent on evaluation and data collection.

The object of the present invention is therefore to provide a method anda device which make it possible to reduce the effort required to achieveas reliable an overall result for view images as possible.

This object is achieved by a method and a device for evaluating at leastone view image with the features according to the respective independentclaims. Advantageous embodiments of the invention are the subject-matterof the dependent claims, the description, and the figures.

In the method according to the invention for evaluating at least oneview image in order to image at least one viewpoint provided in relationto at least one scene shot of a scene and/or at least one view directionof at least one person, for example, a test person, provided in relationto the at least one scene shot, onto a reference, the content of whichmatches at least one part of the scene, for example, locally ortemporally, the at least one scene shot with the at least one assignedviewpoint and/or with the at least one assigned view direction isprovided, and the reference is provided together with a result of the atleast one view image. Furthermore, the at least one view image isevaluated by means of at least one predefined quality measurement and aresult of the evaluation is provided.

The one or even several predefined quality measurements advantageouslymake it possible to evaluate the quality of such a view imageautomatically and thereby quantify it objectively, whereby an objectivestatement about the reliability of the final result of such a view imagecan in turn advantageously be made. Although the reliability of a viewimage cannot be influenced by this directly, the effort to achieve areliable overall result can still be significantly reduced, since theinformation about the quality of a particular view image isadvantageously now known and can be used accordingly. If it is found,for example, that a view image is of high quality, or thatcorrespondingly several view images are of high quality and thus of highreliability, it will not be necessary to collect additional input data,e.g., more scene shots and view data, which reduces overall effortsignificantly. As a result, view images can also advantageously beselected on the basis of their quality, and, for example, only reliableresults of such view images be used for a final evaluation. Above all,however, such an evaluation can be carried out in a fully automated way,so that in comparison thereto, for example, to manually comparingwhether viewpoints have been imaged well or poorly or not at all, aconsiderable time-saving and above all a greater reliability andobjectivity can hereby be achieved as well, because specifying qualitymeasurements makes possible the specification of objective criteria forevaluating the quality of a view image. Such quality measurements can,for example, each evaluate one aspect of the quality of such a viewimage. A quality value according to the quality measurement in questioncan be assigned accordingly to a view image. In this way, individualview images are advantageously made mutually comparable in regard totheir quality, which in turn yields many advantageous applicationpossibilities for utilizing such findings regarding the quality orreliability of the result of a view image.

The scene to which the scene shots relate can represent a real-worldenvironment of the person or even a virtual environment, which is, forexample, viewed with VR (virtual reality) glasses, or an image displayedon a display device or even a superimposition of virtual/digital imagecontents onto the real environment, such as when the scenes are viewedwith AR (augmented reality) glasses.

The at least one scene shot can correspondingly relate to a picture ofthe scenes described above, and, for example, represent a camera shot ofthe environment of the person or the one display on a screen at aspecific point in time as a kind of screenshot so to speak. Thus the atleast one scene shot can represent a picture of a scene by means of oneor more image sensors, a stereoscopic image sensor possibly with depthdata associated with images, an array of image sensors, at least onecamera and other data from one or more sensors, of a 3D scene recordingin which a 3D scene is described as a composition of objects andpossibly of a scene background structure, and/or the recording of afixed or time-varying stimulus (such as a recording of screen contents).Data of a scene shot can also be temporally related to each other. Inthis respect, a scene shot, in particular in connection with acomputer-generated scene or digital image contents of, for example, acomputer-generated, virtual scene, is not necessarily to be understoodas a recording by means of a camera or the capture or storage of acurrently displayed 2D or 3D image of the scene but rather also, forexample, the recording of the scene data of the entirecomputer-generated scene, for example, by storing or providing the scenedata of the computer-generated scene, together with the specification ofa virtual standpoint that defines the view or perspective of the personwith regard to the currently displayed virtual scene. In order todetermine the current perspective, a 3D pose of the person and/or theview direction can, for example, be captured, depending on which ofthese parameters causes a change in the perspective in relation to thescene. In typical VR applications using VR glasses, for example, aperson can look around in the virtually displayed scene by moving hishead and thereby change his view and thus the perspective in relation tothe scene. Such head movements can also be captured accordingly, inparticular together with the corresponding view direction of the personat this point in time, so that these data define the virtual perspectivein relation to the virtual scene. This too can be understood as a sceneshot.

View directions and/or view endpoints or viewpoints can be available aspart of the eye-tracking data with a temporal and/or local relationshipto the scene shot, for example, as a viewpoint onto a scene image and/ora view endpoint or a view direction in a 3D scene. These view data canbe captured and supplied by a mobile eye tracker, for example, an eyetracker that can be worn on the head, or even by a remote eye trackerthat is not head-mounted but attached, for example, to a display device,such as a monitor. This is particularly advantageous when the scene shotrepresents a display shown on the display device so that the view of theuser in relation to the display can be captured in a particularly simplemanner by the remote eye tracker. The scene shot then, for example, nolonger shows the direct field of view of the user (which changes withmovements of the head) but can directly show the presented stimulus,which can in turn represent an image, a video, a stereoscopic video, arendered 3D scene with objects or even as a composition in the AR case.Furthermore, a plurality of (or even no) viewpoints and/or viewdirections can also be assigned to a scene shot. For example, the imagerecording rate at which the scene shots are taken can be less than theacquisition rate at which the eye tracker records the view data, so thata plurality of individual viewpoints and/or view directions wererecorded for a particular scene shot within a specific recording timeinterval. All of these viewpoints and/or view directions recorded inthis scene recording time interval can then be assigned to the sceneshot in question. Similarly, the image recording rate may be greaterthan the acquisition rate at which the eye tracker records the viewdata. If a plurality of viewpoints and/or view directions are assignedto a scene shot (such as a scene image), then all view data orrepresentative view data/datum (median, mean value, otherrepresentation) can be assigned to a scene shot.

The reference can also be present in many different forms. For example,the reference can be a local representation, such as an image or astatic 3D scene, a local-temporal representation, such as a video or adynamic 3D scene, or a content-based/semantic representation, such ascategories/classes of objects acquired as a data structure, or, forexample, as a visual or as a three-dimensional representation ofspecific or prototypical objects.

Such a reference can also have been extracted from a scene shot or fromone of the scene shots. A reference may furthermore be a local, temporalor local-temporal portion of a scene shot. A reference can have beencreated in its particular form (as listed above for scene shots) usingexternal means, and thus independently of an eye tracker.

Furthermore, the view image, which represents the imaging of a viewdirection and/or of a view endpoint or of a viewpoint of a scene shotonto the reference, can be determined manually or automatically.Furthermore, the view image can be determined for each discrete timerecording of a scene shot, for example, image by image of an imagesequence, or for a time interval. The time interval can, for example,correspond to a view event, such as a fixation.

In this respect, the at least one quality measurement can be predefinedor dynamically determinable. For example, the at least one qualitymeasurement can be determined as a function of one or more criteriaand/or can be selected from a plurality thereof by a user.

The viewpoint imaged onto the reference and/or the imaged view directioncan, for example, represent the result of the view image. However, evenan imaging of such a viewpoint which has not been carried out or ismissing can be regarded as the result of the view image.

According to one embodiment of the invention, it can be provided thatthe result of the evaluation is displayed to a user. For example, afterone or more view images have been produced, the reference image togetherwith the imaged viewpoints can be displayed on a display device, whereinthe individual imaged viewpoints are displayed in different colorsdepending on their quality as evaluated in accordance with the at leastone quality measurement, for example, high-quality viewpoints in green,low-quality viewpoints in red.

According to a further advantageous embodiment of the invention, it isprovided that the at least one view image is assigned to at least one ofa plurality of defined quality classes depending on the evaluation onthe basis of the at least one quality measurement. For example, a viewimage can be assigned via a threshold value, for example, to the qualityclass of ‘successful’ or ‘no review necessary’ or even to the qualityclass of ‘to be reviewed/possibly corrected.’ It is also possible, forexample, to evaluate with a quality measurement whether a view directionimage or a view image should have been produced in a scene image andthis view image is however missing, so that such a view image can beassigned to the quality class of ‘missing/possibly to be reviewed/imagepossibly to be determined manually.’ The result of the quality classassignment can in turn be provided and, for example, visualized for auser or output in a different way. For example, the viewpoints can inturn be displayed on the reference with a color corresponding to theirquality class assignment. The quality class assignment can however becommunicated to a user in any other way, e.g., as a visualization in theform of a histogram, which illustrates the numbers of view imagesassigned to the respective quality classes. The quality classes can berepresented on a display device as appropriate icons or menu items, withcorresponding names or designations, such as ‘to be revised,’ forexample. By selection of such a menu item, the view images assigned tothis quality class can then be visualized. In addition, two or three orfour or any number of quality classes can be defined, so that aparticularly differentiated evaluation and examination of the results ofview images is made possible. Furthermore, an algorithm selection may,for example, also be made depending on the quality classes, as describedin the simultaneous application of the same applicant entitled ‘Methodand device for creating a view image,’ filed on the same filing date.Accordingly, the evaluation of the view image can be used to create anew view image in order to improve the quality, this being done on thebasis of an algorithm or at least a part of an algorithm which wasselected in dependence on the quality class of the view image inquestion. In this way, the quality classes can also be used to recreateassigned view images using different algorithms and thus improve theirquality directly, and/or to determine or to influence the type ofalgorithms to be used.

The classification into quality classes also advantageously makes itpossible to carry out an overall evaluation of a view image usingdifferent quality measurements. For example, a view image can beassigned a first quality value in accordance with a first qualitymeasurement, a second quality value in accordance with a second qualitymeasurement, a third quality value in accordance with a third qualitymeasurement, and so on. A mean value, e.g., even a weighted mean value,can be formed from the assigned quality values, wherein the qualityvalues of the individual quality measurements can be given differentweightings. The view image can then be assigned to one of the qualityclasses corresponding to this mean value. This advantageously makespossible an assignment of the view image to one of the quality classeswhile taking into account various aspects of the quality evaluation.

Such aspects, which are evaluated by the quality measurements, are, forexample, the image quality, the image similarity of the environment ofthe viewpoint in the scene image and in the reference, the extent towhich the content of scene images matches the reference in general oreven the matching of the reference and the real scene, and many more.Furthermore, different quality class types can also be envisaged,wherein a particular quality class type covers a plurality of qualityclasses. In this respect, a particular quality class type can beassigned to a quality measurement so that a view image is evaluated inaccordance with a first quality measurement and is assigned to a qualityclass of the first quality class type in accordance with the evaluation,while the view image is in addition evaluated in accordance with asecond quality measurement and, depending on the result of theevaluation, is assigned to a quality class of a second quality classtype, and so on.

Furthermore, the provision of the at least one scene shot with theassigned viewpoint and/or view direction and the provision of thereference with the result of the view image can be effected as provisionof input data, comprise data relating to the at least one scene shot,data relating to the particular viewpoint and/or the particular viewdirection and data relating to the reference as well as data relating tothe result of the view image, and these input data or at least part ofthem can be used for evaluation.

For this reason, in a further particularly advantageous embodiment ofthe invention, the evaluation of the at least one view image by means ofat least one predefined quality measurement is carried out in dependenceon an analysis of the at least one scene shot and/or the at least oneassigned viewpoint and/or the at least one assigned view directionand/or the reference and/or the result of the at least one view image.On the basis of these input data or at least parts thereof, theabove-described aspects to be evaluated in accordance with the qualitymeasurement can advantageously be supplied.

The provision of the at least one scene shot with the at least oneassigned viewpoint and/or the at least one assigned view direction andalso the provision of the reference together with a result of the atleast one view image can in this case also be effected in achronological sequence, e.g., in a streaming process, so that the dataof the at least one scene shot are, for example, received sequentiallyover time and thereby made available. The same also applies to the dataconcerning the reference, the view data and the result of the viewimage. In this case, an analysis of at least parts of these data inorder to evaluate the view image can already begin before all of theseinput data have been received. For example, the sequentially receiveddata of a scene shot can be analyzed immediately after the data arereceived even if not all of the data concerning this scene shot have yetbeen received. Neither do all of the data necessarily have to be usedfor the analysis. Even on the basis of a part of the received data,results or intermediate results can be provided which can be used forfurther processing, without all of the data necessarily having to bereceived and analyzed. In this way, results or partial results may beavailable during processing, in particular even of multiple scene shots,before all of the data, such as those of a single scene shot or ofmultiple scene shots, have been received.

The assignment to a quality class on the basis of the evaluation inaccordance with the at least one quality measurement can in this case becarried out by means of a threshold value method. In other words, if thequality value assigned to the view image in accordance with the at leastone quality measurement, or even the mean value or the value formed froma plurality of quality values, falls within a particular range of valueswhich is limited by one or more threshold values and which has beenassigned to a quality class, the view image will be assigned to thisquality class. The decision regarding the quality class assignment canhowever be made not only on the basis of a threshold value decision, butany classification methods can be considered. In addition to thethreshold value decision, suitable classification methods using a modelalso include, for example, a decision tree, Bayesian classification,logistic regression, a support vector machine, artificial neuralnetworks and/or other methods delivering model-based decisions. Inaddition, so-called classification methods without a model orunsupervised learning methods can be applied for this purpose, such asclustering techniques (for example, k-means clustering, mean-shiftclustering, spectral clustering, affinity propagation, biclustering,hierarchical clustering, and so on), latent variables models,unsupervised artificial neural networks, unsupervised support vectormachines, outlier/novelty detection, matrix factorization (for example,PCA, ICA, NMF, LDA), non-linear dimension reduction (for example, NLDR,manifold learning), or unsupervised linear/Gaussian mixed models. Anycombinations thereof can also be used.

In a further advantageous embodiment of the invention, at least onepredefined action with respect to the at least one viewpoint and/or theat least one view direction is performed in dependence upon theevaluation. Such an action, for example the one described above, canrepresent a visualization of the viewpoints and/or view directions, forexample, with respect to the reference, in which the quality assigned tothe viewpoints according to the evaluation is identified visually. Suchan action can however also be the issue of a notification to the userthat the view images rated as poor should be reviewed or processedmanually. Another first action of this kind can also be the deletion orremoval of view images rated as poor or viewpoints rated as poor fromsubsequent processing steps. In addition, there are many otherpossibilities for such a first action. The first action can furthermorebe carried out not only in dependence upon the evaluation but also, forexample, in dependence upon the quality class to which the view image ofthe respective viewpoints and/or view directions have been assigned.Accordingly, the evaluation-dependent execution of an action withrespect to the viewpoints or view directions on the whole allows notonly detailed and differentiated information about the results of theevaluation to be output to the user but also recommendations for furtherprocessing, up to and including the automatic evaluation-dependentfurther processing of the viewpoints themselves.

In a further advantageous embodiment of the invention, a plurality ofscene shots with their respectively assigned viewpoints and/or viewdirections are provided as the at least one scene shot, and a pluralityof view images for imaging the respective viewpoints and/or viewdirections onto the reference are evaluated by means of the at least onespecified quality measurement. This embodiment is particularlyadvantageous, since an enormous time saving and a reduction in effortare achieved with the invention and its embodiments, especially in thecase of numerous scene shots with numerous corresponding view data andcorresponding view images. In addition, the invention and itsembodiments are particularly suitable for applications in which numerousscene shots and view data have to be evaluated. In this case, therespective view images do not necessarily have to be evaluated by meansof the same quality measurement or quality measurements, but theevaluation of a particular view image can also be performed by means ofdifferent quality measurements. For example, the quality of view imagesfrom different experiments or specific analysis time intervals can, forexample, be investigated from different aspects, which advantageouslypermits many flexible options for adaptation to different situations andapplications.

In a further advantageous embodiment of the invention, metadata areassigned to the at least one viewpoint and/or to the at least one viewdirection and/or to the at least one scene shot and/or the reference.Such metadata represent data going beyond the pure viewpoint data andimage data of the scene shot or of the reference, which provideadditional information on the viewpoint data and/or the scene shot dataand/or the reference data. These metadata can also be provided as partof the input data. For example, an acquisition time can be assigned to aparticular viewpoint. In the same way, this can also be the case for aparticular scene shot. Such metadata may also relate to the tagging ofindividual viewpoint shots or scene shots or the reference. Suchmetadata may also concern the properties of one or more persons, objectsor things involved in the scene shot, such as gender, temperature,membership of a test group, and so on. Such metadata can also concernadditionally recorded data, such as audio data, EEG data or pulse ratemeasurements of the at least one person. Metadata of this kind can alsobe included in the quality evaluation of a particular view image, thusrepresenting a further advantageous embodiment of the invention, if theevaluation of the at least one view image is carried out as a functionof the metadata by means of the at least one predefined qualitymeasurement. The metadata or parts thereof can also be used to determinethe first action which is then carried out in dependence upon theevaluation. It is also particularly advantageous to use such metadata,for example, to make a selection, as will be explained in more detailbelow.

According to a further advantageous embodiment of the invention, atleast one selection of viewpoints and/or view directions of theplurality of viewpoints and/or view directions is made, wherein at leastone predefined second action for the selected viewpoints and/or viewdirections is carried out, in particular wherein the at least oneselection is made in dependence upon the quality classes and/or the atleast one quality measurement and/or the first action and/or a result ofthe execution of the first action. Such a selection can alsoadvantageously be made in dependence upon the described metadata. Thesecond action also can, correspondingly to the above-described firstaction, include a visualization of the results of the view images, withthe corresponding viewpoints once again being color-coded on the basisof their quality, for example. However, according to this exemplaryembodiment, not all of the viewpoints are now visualized but only thosewhich were selected according to the selection. Several such selectionscan also be made and the corresponding actions with these selections canalso be carried out.

Such a selection can now be made according to various criteria. Forexample, in this way, only the viewpoints and/or view directions of aspecific test period or acquisition period can be selected or only theviewpoints of a particular person or group of persons, or of persons ofa particular gender, or similar. To make such a selection, the metadatadescribed above can now advantageously be used. Such a selection canhowever also be made in dependence upon the quality measurements orquality classes. If, for example, a view image is evaluated by means ofseveral different quality measurements, such a selection can alsospecify that only those viewpoints are displayed which were rated asespecially good according to a first quality measurement, or even onlythose viewpoints which were rated as especially good according to thesecond quality measurement. In this way, individual actions can beperformed selectively in dependence upon those aspects of the quality ofthe view images which were specified by the quality measurements inquestion. This has the great advantage that particularly goodpossibilities for adaptation to individual applications and the mostdiverse situations are also given as a result. For example, there may beapplications for which only some aspects of the quality of a view imageare relevant to the final result but other aspects of the qualityevaluation are not. The selection now advantageously makes it possibleto select viewpoints subsequently on the basis of different aspects ofquality, and accordingly to perform the second action only for theselected viewpoints and/or view directions. The selection canaccordingly also be made in dependence upon the quality classes. Forexample, only the viewpoints of a particular quality class can beselected for further processing as a second action and/or viewpoints ofa different quality class can be discarded or deleted as another exampleof such a second action. In addition, the selection can also be made independence upon the first action and/or a result of an execution of thefirst action. For example, the first action may concern the calculationof a total reliability of the overall result of the viewpoint images. Ifthis reliability, for example, does not meet the desired criteria,viewpoints rated as poor can be removed, ignored or deleted whenproviding the overall result according to the selection, and,accordingly, the reliability of the overall result can be recalculatedon the basis of the selected viewpoints and a check can be carried outto see whether the desired criteria regarding the overall reliabilityare now met. In this way, numerous advantageous possibilities areprovided for selecting viewpoints from the totality of all viewpointsand/or view directions, and further processing them suitably. Suchselections can be effected on the one hand in terms of the quality ofthe respective view images but on the other hand according to any othercriteria, which is made possible, for example, by the metadata. Usersthemselves can, for example, also specify criteria for the selection orcan select from a plurality of predefined criteria such criteria thatthe selection is ultimately made as a function of the criterion selectedby the user. In other words, users can thus make selections such thatonly those viewpoints evaluated in accordance with a particular qualitymeasurement are to be displayed, and then those viewpoints evaluated inaccordance with a further quality measurement, or only the viewpoints ofa specific test group, or the like. Selections can also be made fullyautomatically in accordance with specified criteria or rules.

In a further advantageous embodiment of the invention, at least onesequence is specified, which concerns the various quality classes and/orthe various selections of viewpoints and/or view directions and/or theviewpoints and/or view directions within a quality class and/or theviewpoints and/or view directions within a selection, and a third actionis executed for the viewpoints and/or view directions as a function ofthis sequence. For example, it may be provided that the various qualityclasses are specified in a certain order, and, in the case of anevaluation of the result of the view images, the view images assigned toa first quality class are first used for the evaluation, and if, forexample, the result has an uncertainty that is still too high on accountof the insufficient quantity of data, the viewpoints of a second qualityclass are then also used for evaluation, and so on. A sequence within aquality class can also be defined, for example, corresponding to theratings given to the view images within a quality class according totheir respective quality. In this way, those view images rated within aquality class as better can be used first for the third action, forexample, a further processing step for the view images, then those ratedas middling, and so on. The sequence within a quality class can howeverbe defined not only according to the rated quality, but such a sequencemay also represent a chronological order corresponding to theacquisition timepoints of the viewpoints and/or scene shots, or asequence relating to different persons or groups of persons, gender, orthe like. Metadata can also be used accordingly to define such asequence or the sequence definition can be made as a function of themetadata. Such a sequence can also be made in dependence upon thequality measurements, for example, if the evaluation of the view imageis made in dependence upon several quality measurements by means ofseveral quality measurements, the sequence can take into considerationan appropriate weighting or prioritization of some quality measurementswith respect to other quality measurements.

Such a sequence can also be configured between different selections. Ifspecific viewpoints were selected in accordance with a first selection,and additional viewpoints were selected to correspond to a secondselection based on second criteria, a sequence can also be defined forthese selections in order to subsequently carry out the third action independence upon this sequence. Sequences for viewpoints and/or viewdirections within such a selection can also in turn be definedcorrespondingly. In other words, such a sequence can be set up accordingto various criteria or depending on various parameters which can, forexample, also be selectable by a user. The user can therefore, forexample, have the results of the view images visualized, wherein theresults of a test on the current day is displayed first, and then theresults of a test on the previous day, and so on.

An overall result can also be visualized first, then the test results ofa first age class, then the test results of a second age class ofpersons, and so on. The user can also be given a recommendation or aninformation message about view images to be reviewed and/or revised,wherein this recommendation is output first for view images which havebeen rated as comparatively good, then for view images which have beenrated as middling, then for view images which have been rated as poor.This too can be carried out separately for a first age class, for asecond age class, for a first test day, for a second test day, for agender, for a first test scenario, and so on. In this way, particularlyflexible further processing options are provided, which can be used in amanner particularly adapted to individual situations.

The decision regarding the sequence can in turn be dependent onpreviously obtained results, such as also, for example, on the firstaction, the second action or their results. The assignment of a viewimage to a quality class thus depends, for example, on the evaluationvia the quality measurement. The decision about which first action is tobe performed again depends on the evaluation via the quality measurementor on the quality class. A selection of viewpoints and/or viewdirections can in turn depend on their evaluation, on the quality classor on the first action or its result. Furthermore, sequences can bedefined, such as the external sequence between the quality classes orselections, as well as the internal sequence within quality classes orselections, which can in turn depend on the evaluation via the qualitymeasurement itself, on the quality class, on the first action and/or onthe selection. An internal sequence can also be defined, and then anexternal sequence, which in turn depends on the internal sequence. Thethird action can in turn be performed in dependence upon the sequences,wherein the third action itself may in turn be dependent on the qualitymeasurement, the quality class, the first or second action, or theselection. In other words, for a next decision about, for example, thefirst action, the selection, sequences and so on, it is possible to useall previously accumulated and collected information and data, resultsand decisions already taken. Decisions of this kind can however also beinfluenced by further criteria, regardless of those mentioned above, oreven depend on them completely or additionally. In this case, suchcriteria can be provided, for example, by the metadata described.

For this reason, it is an advantageous embodiment of the invention thatthe evaluation and/or the assignment to a quality class and/or the firstaction and/or the at least one selection and/or the sequence and/or thesecond action and/or the third action is determined in dependence uponat least one criterion, in particular in dependence upon the assignedmetadata, for example, upon the acquisition timepoint assigned to aparticular viewpoint and/or a particular view direction. This means thatthe evaluation of the view images by means of the quality measurementcan, for example, also be additionally dependent on such metadata. Thesemetadata can, for example, decide which of several quality measurementsare to be used for the evaluation of the view image or also which aspectof the quality of the view images is relevant to a result and istherefore to be assessed. For example, after a tagging with regard tothe current application area, the metadata can be used to search forrelevant objects during the test, or similar. In some tests, it may onlybe relevant to determine the object a person is looking at, wherein itis irrelevant which part of this object the person is precisely lookingat. In other tests, however, it may be relevant to also determineexactly what part of an object a person is looking at. Small deviationsof the imaged viewpoint from the actual position can already lead to theimaged viewpoint no longer lying on the correct object, while it is alsopossible on the other hand for the imaged viewpoint nevertheless to belying on the correct object despite greater deviations from the actualviewpoint. Whether the magnitude of the deviation of the imagedviewpoint from the actual position is a suitable criterion for assessingquality depends under certain circumstances on the situation or on theobjective of a test. If it can, for example, be deduced from themetadata that a very specific target object is relevant in theexperiment, a quality measurement can be used which assesses whether theviewpoints lying on particular objects in the scene shots were alsoimaged onto the corresponding objects in the reference, regardless ofwhether these were also imaged onto the correct area of such an object.In other cases, however, a quality measurement can be selected whichassesses the quality of the view images with regard to whether theviewpoint was also imaged onto the correct area of an object. Metadatacan thus also be used advantageously for laying down criteria for theassessment of the quality of the view image and selecting appropriatequality measurements for assessing the quality of the view images.Metadata can also be used accordingly for defining a sequence, forexample, for different groups of people, or a chronological sequence, inorder to define a selection or even to decide about various actions,such as the first, the second and/or the third action, which are to beapplied to a particular quality class, selection, or sequence.

Assessment or evaluation of the view images by means of one or morequality measurements thus advantageously makes possible anobjectification of the assessment of the quality while taking intoconsideration different situations, objectives and/or prespecifiableaspects of the quality. This advantageously makes it possible to statean overall reliability of the ultimately provided overall result of theview images with regard to the aspects of quality relevant to theparticular application case.

Various possible quality measurements are explained in more detailbelow. As described at the beginning, at least a part of the input datais used to evaluate the view images since, above all, the comparisonbetween the positions of the viewpoints and/or view directions in aparticular scene shot contains the most information about the quality ofthe view image as compared with the result of the view image on thereference. However, there are also aspects of the quality which can, forexample, be measured solely with the aid of the scene shots, such as theimage quality, for example. There are also aspects of quality which canbe measured completely independently of the scene shots and thereference, such as the number of viewpoints present in relation to sceneshots in comparison with the number of imaged viewpoints or the numberof unimaged and therefore missing viewpoints.

According to an advantageous embodiment of the invention, a degree towhich the content of the reference matches the at least one part of thescene and/or the content of the at least one scene shot matches the atleast one part of the scene is evaluated in accordance with the at leastone quality measurement. There are situations in which a reference isprovided first, for example, in the form of a schematic drawing ordiagram or even in the form of a computer-aided design, which is then tobe recreated or reproduced as a real scene. In such a case, there may bedeviations between the real scene and the reference; for example, it mayhappen that objects in the real scene do not have the same distances orpositions relative to each other as in the reference. This can in turnlead to incorrect images of viewpoints on the reference in relation tothe scene shots. In order to determine whether and how well thereference matches the scene, the content of the reference can, forexample, be compared with the content of the scene shots. If groups ofobjects are in a configuration in the scene shots which differs fromthat in the reference, it can, for example, be assumed that thereference deviates from the real scene. Even the extent of suchdeviations can be quantified by means of image analysis methods.

This quantification can be provided for a particular view image as aquality value based on the quality measurement measuring the extent towhich the content of the reference matches the at least one part of thescene. The situation is similar for the quality measurement forevaluating an extent to which the content of the at least one scene shotmatches the at least one part of the scene. During a test, it may happenthat a person becomes distracted and is therefore not looking at therelevant scene but rather, for example, at the floor, at other people,out of the window or the like. In the case of a head-mounted scenecamera, this results in such scene shots not including any parts of theactually relevant scene. Such discrepancies between a scene shot and theat least one part of the scene can also be easily determined by imageanalysis methods. To this end, the scene shots are, for example,compared with each other and also with the reference. If there is acontent match between the reference and other scene shots, for example,it can be assumed that this content match represents the relevant scene.If some of the scene shots do not have this content match, or onlypartially, it can be inferred from this that there is a deviation of thecontent match of the at least one scene shot from the at least one partof the scene. This deviation can also in turn be quantified by thequality measurement and provided as a corresponding quality value forthe view image of the viewpoint provided in relation to this scene shot.If, for example, there is no content match between the scene shot andthe relevant scene, there will also not be any content match with thereference either. It will therefore not be possible to image onto thereference a viewpoint provided in relation to this scene shot. If thisviewpoint is finally classified as a missing viewpoint or missing viewimage, this view image can be assigned in particular to the qualityclass ‘not imaged, correct.’ If, on the other hand, there is a contentmatch between the scene shot and the reference, especially in the areaof the viewpoint in relation to the scene shot, and if such a viewpointis nevertheless not imaged, such a view image can, for example, beclassified as ‘not imaged by mistake/manual correction necessary.’

According to a further advantageous embodiment of the invention, asimilarity is evaluated on the basis of the at least one qualitymeasurement between a first predefined image area, which can inparticular be a 2D image area or also a 3D image area, around theviewpoint assigned to the at least one scene shot and/or the viewdirection in the at least one scene shot and a second predefined imagearea, which can also be a 2D image area or also a 3D image area, aroundthe corresponding imaged viewpoint and/or the view direction in thereference. The first and second predefined image areas in this caserepresent only a partial area of the overall image of the scene shot andthe reference. This has the great advantage that in the case of a checkof such a local content match, all of the images of the scene shot andthe reference do not need to be examined and analyzed. This can save anenormous amount of computing power and time. In this case, it is aboveall the local match between the areas around the viewpoint in the sceneshot and around the imaged viewpoint in the reference which isparticularly suitable for making a statement about the accuracy of theview image and thus about its quality. This similarity can in turn alsobe quantified by the quality measurement and a corresponding qualityvalue can be assigned to the view image. The better the match betweenthese local image areas, that is, between the first predefined imagearea and the second predefined image area, the higher the quality can berated and, correspondingly, the higher the quality value can turn out.

There are various ways of determining this similarity between the firstand second predefined image areas. Accordingly, different qualitymeasurements can also be used for the evaluation of this similarity.

For this reason, it is an advantageous embodiment of the invention thatthe evaluation is carried out in accordance with the at least onequality measurement, in particular the evaluation of the similarityand/or match between the first and the second predefined image areas,based on a brightness and/or color comparison and/or comparison of edgesand/or object comparison of detected objects and/or gradient comparisonin terms of a color gradient and/or brightness gradient and/or contrastgradient. A similarity comparison or the determination of the degree towhich the two image areas match can in this case generally be carriedout on the basis of known and well-established image analysis methodsand/or content analysis methods. Some other examples of such methods arethe assignment of extracted image features (statistical description, forexample, with the histogram of the properties of color, brightness,edges, and so on around a location), such as point features, and/or theconsideration of disparity, curve, gradient; depth(s), curve, gradient;contrast, curve, gradient; texture properties; frequency analysis and/orspectral decompositions (wavelet, DFT, and so on).

The evaluation of similarity can also be carried out on the basis of allthese criteria mentioned, wherein each of these criteria defines acorresponding quality measurement. In order to define a correspondingquality measurement, it is also possible to select only one sub-group ofthese criteria mentioned. A single quality measurement can also beprovided, which is used to assess the quality value of all these namedcriteria for evaluating similarity. One key aspect of quality is aboveall how well a scene image is imaged locally onto the reference imageand how this image of the scene image differs, for example, in thebrightness values of the viewpoints, from the relevant portion of thereference, that is, the portion of the reference image which includesthe imaged viewpoint. A view image (such as a projective transformation)can thus be applied to the associated scene image and the pixeldifference can be calculated as a 2D sum of the absolute differencebetween the brightness values of the individual viewpoints of theprojected scene image and of the reference image. Another qualitymeasurement describes, for example, the edge structure in a projectedscene image and compares it with the edge structure of the reference.Another quality measurement can use object detection and/or objectrecognition in order to evaluate and/or compare in a scene shot and in areference the similarity of the objects found and/or located in thevicinity of the original and of the imaged viewpoint and/or viewdirection. Another quality measurement can evaluate a view directionimage or view image on the basis of the intensity of the movementbetween scene images for a particular assignable view direction image.As a result, numerous advantageous possibilities are provided forevaluating the quality of a view image.

A fundamental problem with the restriction to a local analysis areaaround the viewpoint and the imaged viewpoint is however that with alocal matching of the scene shot and the reference, it cannotnecessarily be assumed that the viewpoint in relation to the scene shotwas also correctly imaged onto the reference. In studies relating toshelves, for example, in which a shelf is holding many similar oridentical objects, such as in the case of a supermarket shelf with aplurality of identical products arranged alongside each other, arestriction to a local area which has been dimensioned too small canunder certain circumstances not be adequate for evaluating the qualityof the view image. If, for example, several identical bottles arearranged on a shelf and the person is looking at one of them, and if nowthe viewpoint is imaged onto one of these bottles in the reference, itmay be the case that a local comparison between the scene image segmentand the reference segment around the viewpoint in question results in amatch but that the viewpoint was nonetheless imaged onto this one ofthese several bottles. It can thus under certain circumstances be veryadvantageous to consider a larger area around the viewpoint in the sceneimage and the imaged viewpoint in the reference in order to exclude suchmix-ups. However, an enlargement of the first and/or second areas doesin turn mean more computational effort.

For this reason, a particularly advantageous embodiment of the inventionis when the first and/or second predefined image areas are defined inaccordance with at least one predefined criterion, in particular independence upon a user input and/or a specified minimum accuracy and/oran intermediate result of the evaluation of the similarity and/or animage analysis of the scene shot and/or of the reference and/or as afunction of the metadata. This advantageously permits the size of thepredefined first and second image areas to be selected to suit thesituation. Thus, on the one hand, this choice can be left to the userhimself, who is aware, for example, that such mix-ups can happen in thepresent experiment or then again maybe not. Furthermore, a minimumaccuracy or minimum reliability can also be specified for theevaluation. If, for example, a very low reliability is selected for theevaluation, the first and/or second predefined areas can be selectedsmaller while if, on the other hand, a higher minimum accuracy isrequired, the first and/or second predefined areas can be selectedlarger. It is also particularly advantageous to select the size of thepredefined image areas in dependence upon an intermediate result of theevaluation of similarity and/or an image analysis of the scene shotand/or the reference. If, for example, the evaluation of similarity onthe basis of a very small first and/or second predefined image areareveals that they match, the evaluation of similarity on the basis of alarger selected first and/or second predefined image area canadditionally be examined in order to verify the result. If, on the otherhand, it is already determined on the basis of the comparison of thefirst and/or second predefined image areas which were selected to bevery small that they do not match, a further comparison or check forsimilarity using a larger selected image area can be dispensed with. Animage analysis of the scene shot and/or the reference is also helpful inselecting the size of the predefined image areas. For example, a singlescene shot and/or the reference can be examined to see whether the sceneincludes any similar objects with which there is a likelihood ofmix-ups. If this is not the case, the first and/or second predefinedimage areas can be selected smaller for the evaluation of the viewimage. Knowledge of whether similar objects are present in the scene canadvantageously be obtained from an analysis of a very small number ofscene shots and/or the reference. This information can also be obtainedfrom metadata in the same way. If these metadata, for example, due totagging, provide information about the test scenario, such as by the tag‘shelf study,’ this information can be used to suitably choose the firstand/or second predefined image areas. In the case of a shelf study, forexample, it is to be assumed that the scene in question has severalsimilar objects. However, in a study of human interaction, it canhowever be assumed that due to the individuality of the appearance ofpersons, there is no risk of such confusion and, accordingly, the firstand/or second predefined image areas can be selected smaller.

In the same way, it is also advantageous, for example, to define thedescribed quality measurement for evaluating similarity in dependenceupon this predefined criterion or the predefined criteria describedabove. If, for example, there is no risk of confusion between objects,similarity can be evaluated on the basis of a brightness and/or colorcomparison of the predefined image areas. Such a brightness and/or colorcomparison represents a particularly simple and time-efficient way ofmaking a comparison. If, on the other hand, there is a risk ofconfusion, as an alternative or addition to this brightness and/or colorcomparison, an edge comparison and/or an object comparison of detectedobjects, for example also with regard to their positions relative toeach other, can be used for the evaluation, which correspondinglysignificantly increases the reliability of the evaluation.

Through these measures, not only the first and/or second image areas butalso the quality measurements to be used for the evaluation canadvantageously be selected to suit the situation in question so that,despite the localization problem, a sufficiently high reliability canalways be provided without having to select the predefined image areasunnecessarily large, whereby an enormous time saving can in turn beachieved.

According to a further embodiment of the invention, a presence orabsence of imaged viewpoints and/or view directions is evaluated inaccordance with the at least one quality measurement. If, for example, avery great number of view directions and/or viewpoints were not imaged,it can also be concluded that the overall result of the view images isto be rated overall as relatively poor. In this case, there are alsomore possibilities for differentiation, namely whether the lack ofviewpoints is due to the fact that viewpoints could not be imagedbecause there are no corresponding objects between scene shot andreference, or whether corresponding objects were merely not detected bymistake.

According to a further embodiment of the invention, a statisticalvariable determined from the viewpoints and/or view directions inrelation to the scene shots and/or the imaged viewpoints and/or viewdirections is evaluated in accordance with the at least one qualitymeasurement. For example, in this regard, the distribution of theviewpoints imaged onto the reference can be examined and a quality ofindividual and also of the totality of imaged viewpoints can be derivedtherefrom. For example, a marked scattering of the imaged viewpoints mayindicate a poor image quality. If most of the imaged viewpoints areconcentrated in a particular section of the reference image and only afew imaged viewpoints fall outside this section, these aberrantviewpoints can be classified as outliers or rated with a low qualityvalue. Statistical intermediate steps or other image and signalprocessing steps can also be used for evaluating quality and defining acorresponding quality measurement, and thus also transformations,partial statistics, correlations and so on, and combinations thereof.

According to a further advantageous embodiment of the invention, animage quality of the at least one scene shot and/or the reference isevaluated in accordance with the at least one quality measurement. If,for example, the image quality of a scene shot is very poor, forexample, due to so-called motion blur, in other words, the smearing orentanglement of such a recording caused by an excessively fast cameramovement, it must also correspondingly be assumed that even the positionof the viewpoint in this scene can only be determined very imprecisely[,that] object detection techniques do not give good results on the basisof such poor images, and that the imaged viewpoint is also accordinglyto be rated as being of a lower quality.

According to a further advantageous embodiment of the invention, therelevance of imaged and/or not imaged viewpoints and/or view directionswith regard to a predefined target object or a predefined objective isevaluated in accordance with the at least one quality measurement. If anexperiment aims at investigating in respect of a quite specific targetobject, viewpoints which are not already on the target object in sceneshots will not in any case be relevant to the test result. Accordingly,the evaluation of the quality of such view images is also irrelevant tothe final result. Accordingly, it is advantageous to provide a qualitymeasurement which evaluates the relevance of such viewpoints with regardto a predefined target object. For example, information about relevanttarget objects can in turn be derived from metadata, be predefined by auser, or the like. Accordingly, this also applies to a predefinedobjective which can correspondingly define which target objects ortarget areas of a scene are relevant and which are not.

In a further advantageous embodiment of the invention, a plurality ofdifferent quality measurements is specified for the at least one viewimage and the assignment of the at least one view image to at least oneof the plurality of defined quality classes is carried out in dependenceupon the evaluation by means of the plurality of quality measurements.This advantageously makes it possible to quantify different aspects ofthe quality of a view image by means of the corresponding qualitymeasurements. The use of such different quality measurements is, forexample, also advantageous when some aspects of quality are not at allrelevant to a particular objective or particular test. However, it isthen left up to the user, for example, to make or specify an appropriateselection of viewpoints, a sequence or similar in dependence uponvarious acquired aspects of quality which were quantified by thecorresponding quality measurements. In this way, the results of the viewimage can ultimately also be visualized under various of these acquiredaspects and thereby be made comparable.

Furthermore, the invention concerns a device for evaluating at least oneview image for imaging onto a reference at least one viewpoint providedin relation to at least one scene shot of a scene and/or at least oneview direction of at least one person provided in relation to the atleast one scene shot, said reference having a content match with atleast one part of the scene. In this case, the device has an interface,via which the at least one scene shot with the at least one assignedviewpoint and/or the at least one assigned view direction and thereference can be supplied to the device along with a result of the atleast one view image. Furthermore, the device comprises an evaluationunit that is designed to evaluate the at least one view image by meansof at least one predefined quality measurement and to provide a resultof the evaluation.

The advantages mentioned in regard to the method according to theinvention and its embodiments apply in the same way to the deviceaccording to the invention. In addition, the method steps described inconnection with the method according to the invention and itsembodiments make possible the further development of the deviceaccording to the invention through further concrete features.

The invention also includes a control unit for the device, in particularfor the evaluation unit. The control device has a processor unit that isset up to produce an embodiment of the method according to the inventionor one of its embodiments. For this purpose, the processor unit cancomprise at least one microprocessor and/or at least onemicrocontroller. Furthermore, the processor unit can include programcode that is configured to produce the embodiment of the methodaccording to the invention when the program code is executed by theprocessor unit. The program code can be stored in a data memory of theprocessor unit.

Further features of the invention result from the claims, the figuresand the figure description. The features and combinations of featuresmentioned above in the description and also the features andcombinations of features mentioned below in the description of thefigures and/or shown only in the figures can be used not only in thecombination indicated in each case but also in other combinationswithout departing from the scope of the invention. Embodiments of theinvention which are not explicitly shown and explained in the figures,but which by way of separate combinations of features arise from thedescribed embodiments and can be produced are thus also to be regardedas included and disclosed. Embodiments and combinations of featureswhich thus do not have all the features of an originally formulatedindependent claim are also to be regarded as disclosed. Embodiments andcombinations of features which go beyond or deviate from thecombinations of features described in the references of the claims areto be regarded as disclosed, in particular by the above-describedembodiments.

The figures show:

FIG. 1 a schematic representation of a scene and a person viewing thescene;

FIG. 2 a schematic representation of a scene shot with a viewpointrelating to the scene shot and a reference in accordance with anexemplary embodiment of the invention;

FIG. 3 a schematic representation of a scene shot with a viewpointrelating to the scene shot and a reference onto which the viewpoint wasincorrectly imaged, in accordance with an exemplary embodiment of theinvention;

FIG. 4 a flow chart illustrating a method for producing a view image inaccordance with an exemplary embodiment of the invention; and

FIG. 5 a schematic representation of a device for evaluating a viewimage in accordance with an exemplary embodiment of the invention.

The exemplary embodiments described below are preferred embodiments ofthe invention. In the exemplary embodiments, the components described ofthe embodiments are in each case individual features of the inventionwhich are to be considered independently of each other, which in eachcase also develop the invention independently of each other and thus arealso, either individually or in a different combination than the showncombination, to be regarded as a constituent part of the invention.Furthermore, the described embodiments can also be supplemented byfurther features of the invention that have already been described.

In the figures, elements with the same function are designated by thesame reference symbols.

FIG. 1 shows a schematic representation of a person 10 who is currentlyviewing a scene 12. In this case, the person 10 is wearing glasses 14with an integrated eye tracker which is continuously capturing view dataof the person 10, while the person 10 is viewing the scene 12.Furthermore, the glasses 14 have a scene camera 16 which meanwhile isalso continuously recording images of the scene 12. In this case, theacquisition of the view data is temporally matched to the acquisition ofthe scene images or is set or settable in relation to these sceneimages. For example, the acquisition of particular view data or viewdirections or viewpoints of the person 10 determined therefrom and therecording of a particular image at a particular time recording can takeplace synchronously, or the acquisition of the view data and of theimage recordings can be provided with a time stamp, so that a particularviewpoint or a particular view direction can in each case be assigned toprecisely one scene shot. An example of such a scene shot S is shown inFIG. 2 and FIG. 3.

Here, FIG. 2 shows a schematic representation of a scene shot S of thescene 12 from FIG. 1 with a viewpoint B of the person 10, which wasdetermined on the basis of view data acquired at the time of the sceneshot S, as well as with a reference R in order to illustrate a viewimage in accordance with an exemplary embodiment of the invention. Foreye-tracking data, in which view directions and/or view endpoints B areassigned to a time recording of the scene observed by the person 10observed by the eye tracker, the image of the view directions and/orview endpoints B onto a reference R can be very helpful, e.g., when aplurality of recordings of such eye-tracking data is to be madecomparable in relation to a reference R.

The scene shot S here represents an example of a recording of the scene12 which was made by the scene camera 16 at a specific point in time.Furthermore, a viewpoint B assigned to this scene shot S was calculatedon the basis of the view data of the person 10 which were acquired bythe eye tracker, said viewpoint B also being shown in the scene shot S.In this example, the reference R also represents an image recording ofthe scene 12. The reference R can, for example, be one of the sceneshots S, a section of one of the scene shots S, or even a separatelyrecorded image of the scene 12, even one such as recorded with a cameraother than the scene camera 16 worn by the person 10. According to theview image W, the viewpoint B in relation to the scene shot S is nowimaged onto a corresponding viewpoint B′ in relation to the reference R.With such a view image M, numerous viewpoints B, which are present inrelation to numerous scene shots S, can in particular be imaged onto acommon reference R, whereby the comparability of the acquired viewpointdata can be considerably improved, for example. In order to perform sucha view image M, certain algorithms may be used. Such an algorithm can,for example, reference the scene shot S to the reference shot R andobtain therefrom, for example, a transformation which images the sceneshot S onto the reference R. This transformation so obtained can then beapplied to the viewpoint B determined in relation to the scene shot S,correspondingly supplying the corresponding imaged viewpoint B′ on thereference R. The referencing between scene shot S and the reference Rcan, for example, be carried out on the basis of simple image analysismethods. Alternatively or additionally, methods for object detectionand/or object classification can also be used.

The scene shot S can in general be available in the most varied forms,for example, as a 2D recording or even as a 3D recording that wasrecorded, for example, using stereo cameras. It can also be therecording of a purely virtual, for example, computer-generated scene, oralso the recording of an AR scene, and so on. The viewpoint data canalso be available in the most varied forms, for example, as 2Dviewpoints or 3D viewpoints or even as 3D view directions in a 3D scene,and so on. The reference R can also take the most varied forms. Forexample, the latter may be present in the form of different definedobject classes which classify different objects. A first object classcan, for example, relate to bottles, a second object class to boxes, athird object class to cans, and so on. In order to produce a view imageM on the basis of such a reference R, algorithms that work on the basisof object classifications are especially suitable. For this purpose, forexample, the scene shot S can be examined for objects of these objectclasses and it can be checked to see whether a particular viewpoint Bpresent in relation to such a scene shot S is positioned on an objectthat is assigned to such an object class. If this is the case, as isillustrated in FIG. 3 for the viewpoint B resting on the bottle 18, thisviewpoint B according to the view image M can be assigned accordingly tothe first object class for bottles. Such a view image M can also beproduced manually by a user himself.

In all of these cases, errors can occur so that the viewpoint B isimaged onto the wrong position in the reference R or even, under certaincircumstances, not imaged at all. This is illustrated in FIG. 3.

FIG. 3 shows again the scene shot S as well as the viewpoint B inrelation to the scene shot S, and also once again the reference R withthe viewpoint B′ imaged according to the view image M onto the referenceR. In this example, the viewpoint B in the scene shot S was imaged ontothe wrong bottle of the bottles 18 in the reference R. Such incorrectimages ultimately negatively affect the overall result that is to beobtained on the basis of such view images. However, the bigger problemis that it is not known how reliable the view images are from which thefinal result was obtained, so that ultimately the reliability of thefinal result is also unknown.

The inventions and their embodiments now solve this problemadvantageously by the quality of the view images M or their result, thatis, for example, the imaged viewpoint B′, being evaluated in accordancewith at least one specified quality measurement. On the basis of such anevaluation, view images can be assigned to a quality class. Thisadvantageously makes it possible to quantify the quality of view imagesand to ultimately specify therefrom the reliability of a result whichwas obtained from the view images. In this way, an informativestatistical description of the quality of view direction images andviewpoint images is made possible in a simple way for large parts or allview images. Numerous other advantageous applications also becomepossible due to the assignment to quality classes as well as due to theevaluation itself. For example, only highly rated view images can beused to calculate a result, or the user can be notified of possiblydefective view images and check them manually.

Results that may be obtained from such view images are, for example:which object in a scene was viewed most often and which the least; whichobject first drew the attention of a person and which object did so lastor not at all; whether a specific target object was looked at or not; orhow often or for how long; whether objects are looked at in aconcentrated or focused manner or only fleetingly; and much more. Suchobservations can then be made in relation to several different persons,and, by means of comparisons, the corresponding findings can beobtained, for example, that a particular object in a scene especiallydraws the attention of persons in a certain age class while otherobjects draw the attention of persons in a different age class. Suchexperiments can also be carried out repeatedly for a single person inorder to analyze or investigate, for example, a learning advance ordevelopment on the basis of the comparison of the view images for theexperiments in question. General fields of application for the presentinvention and its embodiments are, therefore, generally to be found inthe sphere of the evaluation of eye-tracker data, such as the (off-line)post-processing of recordings in order to evaluate and improve thequality of mobile eye-tracking experiments/studies, for instance, forthe purposes of research into user friendliness or suitability for use,for development psychology or market research, as well as of remoteeye-tracker experiments/studies concerning the measurement of viewbehavior in the case of varying contents as a stimulus, for instance,internet browsing, UI/application dialogs, usability tests, and also foron-line/use of the quality evaluation or of the eye-tracker datafiltered by the quality evaluation for control (as an action), such asthe selection of stimuli or the display, further processing, storage offurther analysis results which are supported by the view directionimaged.

Numerous different criteria come into consideration in this case asquality measurements for evaluating a view image. By means of thequality measurement, an evaluation is made as to how well the positionof the imaged viewpoint or view direction of the person agrees with theactual, real position of the viewpoint or view direction; in otherwords, the extent of this matching and/or an extent of the matchingbetween the object viewed according to the viewpoint imaged onto thereference and the object actually being looked at by the person. Withregard to the above-mentioned variables—in other words the extent—thequality measurement specifies a quality value for a view image, whichvalue represents an estimated value or an approximation for this extent.In particular, different quality measurements can be predefined in thiscase, which can themselves be predefined in particular by the device 22(see FIG. 5) for the evaluation of the view image or be dynamicallyselectable by the device 22 according to predetermined selectioncriteria, which can also be specifiable by the user himself, andquantify the correspondingly different aspects of the quality of a viewimage M, that is, of the image of a viewpoint B and/or view direction ofa scene shot S on the reference R.

One particularly advantageous and suitable quality measurement is, forexample, the similarity between a first predefined image area, such asin the scene shot S the image area 20 s (shown here) around theviewpoint B assigned to the scene shot S, and a second predefined imagearea, such as the image area 20 r (shown here) around the correspondingimaged viewpoint B′ in the reference R. The similarity between thesespecific image areas 20 s, 20 r can be determined according to variouscriteria, e.g., on the basis of a brightness and/or color comparisonand/or edge comparison and/or object comparison of detected objects andtheir positions relative to each other. In this case, all of thesecomparison criteria can also define each of the individual qualitymeasurements. At the same time, it is a further advantage if the size ofthe predefined areas 20 s, 20 r is adaptable or selected according tocertain criteria. The smaller the predefined areas 20 s, 20 r are, themore time-efficiently and more effectively the similarity of the imageareas 20 s, 20 r can be checked. However, especially when a scenecontains a very great number of objects that are confusable and similarin appearance, such as in this shelf example in FIG. 3, it isadvantageous to select the predefined areas 20 s, 20 r larger. If, inthis example, the predefined areas 20 s, 20 r were selected too small, amatch in the similarity of these areas would be detected locally, sincethe two illustrated bottles 18 are similar at least locally. If, on theother hand, a larger area, as shown here, is examined, it is determinedthat these areas 20 s, 20 r have differences between each other.

The size of such an area 20 s, 20 r can, for example, be specified by auser, determined in dependence upon an analysis of a single scene shotS, in dependence upon the analysis of the reference R, from which it canbe determined, for example, whether the scene 12 includes similarobjects; even metadata can be used which, for example, provideinformation about the objectives or scope of an experiment which can,for example, be determined via a tag, such as ‘shelf study.’

The quality measurement ‘image similarity’ thus compares the similarityof the environments of the viewpoints B, B′ between scene image S andreference R. By means of the image similarity, view images M are, forexample, classified by the evaluation unit 24 of the device 22 (see FIG.5) via threshold values into the quality classes of ‘successful/donothing’ and ‘to be reviewed/possibly corrected.’ In addition, however,there are numerous other quality measurements that can be used for theevaluation of view images M. Using a further quality measurement—whethera view image M should have been made within a scene image S—it isdetermined whether these view images M are lacking and accordinglyassigned to the category ‘missing/possibly to be reviewed/possiblydetermine image.’ The remaining images M are assigned to the qualityclass ‘no images/nothing to be done.’ Correspondingly, a user can begiven suggestions as to which view images M are to be reviewed and whichare not. For reviewing, the user can be shown, for example, the sceneimage S in question with the viewpoint B and the reference R and, ifpresent, the viewpoint B′ imaged onto the reference R.

However, in addition to these quality classes, any number of otherquality classes can be defined as well. Another example of this is thequality class ‘review time interval edges.’ In the usual experiments,persons, such as test persons, are normally not looking permanently atthe relevant scene. In addition, a briefing of the test persons can beheld before such experiments and also a discussion of the experimentafterwards, wherein the scene camera 16 in the meantime continues totake scene shots and the eye tracker continues to determine the viewdata of the test persons. Ultimately, however, such scenes and view dataare not relevant to the evaluation of the test results. In this case,the relevant time intervals can also be determined automatically by thedevice 22. For example, this can be done through a comparison betweenthe scene shots S and the reference R. If there is no content matchwhatsoever between scene shots S and reference R, it can be assumed thatthese scene shots S do not yet form part of a relevant analysis timeinterval. In other words, if no time interval has been fixed fordetermining the view images M but these must instead be determined asdescribed, reviewing the edges of time intervals of the view directionimages or view images M resulting from the analysis is helpful inevaluating the correctness of the time intervals or defining themcorrectly. For this reason, the specification of the limits of a timeinterval via view direction images or view images M can itself be usedas a quality measurement and these view images M can be assigned to thequality class ‘review time interval edges.’ In other words, all viewimages M which fall within a predefined temporal proximity of such anautomatically determined time interval edge can be assigned to thisquality class.

In this case, the time interval can even have gaps, hence be interruptedby missing view images M and nevertheless be evaluated as a timeinterval with respect to the assignment to the quality class ‘reviewtime interval edges.’ The maximum number of missing view images M to betolerated can be a second parameter of the classification for thisquality class, i.e., a further quality measurement. These aforementionedgaps, or images missing between view images M, are on the other handassigned to the quality class ‘missing/possibly to be reviewed/possiblydetermine image.’ The remaining images are assigned to the quality class‘no images/nothing to be done.’

In this way, following such a quality class assignment, an analysissoftware program of the device 22 can also, for example, display thequality classes with their names and, as a text, a correspondingsuggestion of an action for the user. The evaluation in accordance withthe quality measurements as well as the quality class assignment makenumerous other applications possible as well, as will be described inmore detail below with reference to FIG. 4.

FIG. 4 shows a flow chart illustrating a method for evaluating a viewimage M in accordance with an exemplary embodiment of the invention. Tothis end, at least one quality measurement GM is determined first instep S10. This quality measurement GM or even these quality measurementsGM can be predefined and retained in every repeated execution of themethod; they can however also be set dynamically as a function ofvarious criteria. As is shown here, the at least one quality measurementGM can be determined, for example, as a function of the input data, thatis, the scene shots S, the reference R, the viewpoint data B, theviewpoints B′ which may have been imaged, as well as metadata MDoptionally provided together with the input data. After specification ofthe quality measurement GM in step S10, at least a portion of the inputdata is analyzed in step S12 in order for a particular view image M tobe evaluated on this basis in accordance with that particular qualitymeasurement GM. As the result of such an evaluation, a quality value GWfor a particular quality measurement GM can be assigned in step S14 to arespective view image M. On the basis of this at least one quality valueGW, the relevant view image M can be assigned to a quality class GK of aplurality of defined quality classes. As an additional option, such anassignment can also be made as a function of the metadata MD provided.This quality class assignment is in this case carried out in step S16.In step S18, a first action A1 can now be performed as a function of theassigned quality class GK. For example, such a first action A1 may beprovided in the visualization of the imaged viewpoints B′ on thereference R together with a visualization of their quality. However,such a first action can also represent any other further processing stepfor the further processing of the results of the view image M. Theselection of such a first action A1 can in addition also depend in turnon the metadata MD provided as well as on the quality values GWthemselves. Furthermore, in step S20, a result E1 of the execution ofthis first action A1 is output. Alternatively or additionally to such afirst action A1, a selection AW of view images and/or imaged viewpointsB′ can also be made in step S22 and a second action A2 can be performedin step S24 in dependence upon this selection AW. This selection AW can,in turn, be dependent on the quality values GW, on the quality classassignment GK, or on the metadata MD. For example, the highest-ratedview images M can be selected according to their quality classassignment GK; only those view images M with the highest ratingaccording to a specific aspect of quality can also be selected accordingto the corresponding quality measurement GM, i.e., on the basis of theassigned corresponding quality value GW. The selection AW can alsorepresent a particular group of individuals or also a time-relatedselection which is thus made in dependence upon the metadata MD. If thefirst action A1 was carried out beforehand, the selection AW can also bemade in dependence upon this first action A1 and/or upon a result E1 ofthis first action A1. In the same way, the second action A2, which iscarried out with the selection AW which was made, is in turn selectedfrom the quality values GW and/or depends on the quality class GK and/orthe metadata MD. For example, this second action A2 can include avisualization of only the selected view images M or imaged viewpoints B′on the reference R according to their quality class assignment GK orcorresponding quality measurement GM. Furthermore, the result of thissecond action A2 may be provided in turn in step S26 as result E2. Inaddition, a sequence RF can be defined in step S28. Such a specificationof a sequence can, for example, be made directly after the assignment toa quality class GK in step S16 and can accordingly relate to a sequenceRF of the quality classes GK, as an external sequence RF, so to speak,and to a sequence RF within the quality classes GK of the individualview images M, as an internal sequence RF, so to speak. In the eventthat a selection AW was previously made in step S22, the sequence RF canalso relate to this selection AW. If, for example, multiple selectionsAW were made, the sequence RF can in turn define an external sequence RFof these respective selections AW, as well as alternatively oradditionally also an internal sequence RF of the individual view imagesM of such a selection AW. In addition to the quality class GK and theselection AW, the definition of this sequence RF, that is, not only theinternal but also the external sequence RF, can in turn be effected as afunction of individual quality values GW; as a function of metadata MD,for example, in order to define a temporal sequence RF; and also as afunction of the optional first action A1 and/or its result E1; and inthe same manner also as a function of the second action A2 and/or itsresult E2. Depending on such a defined sequence RF, a third action canin turn be performed in step S30, which can furthermore also beoptionally dependent on the other variables, i.e., the quality valuesGW, the quality classes GK, the metadata MD as well as the second actionA2 and/or its result E2. Furthermore, the result E3 of this third actionA3 is in turn provided in step S32.

In this way, following the assignment of the view images to therespective quality classes GK, different selections AW, sequences RF oractions A1, A2, A3 can be performed, wherein a particular subsequentstep of this kind can in turn be dependent on the information providedin previous steps and/or as input data. The method and the device 22 canthus bring about a decision in each case not only on the basis of one ormore quality measurements GM or possibly one or more assignments toquality classes GK but also on the basis of previously reached decisionsand/or additional criteria and/or intermediate results. In addition tothe dependences shown here by way of example, other criteria can beprovided for the decision concerning the particular steps. Such criteriamay represent variables or data going beyond the quality measurements ofview direction images, such as the metadata that can be used to bringabout a decision. Criteria are variables/data or derived, combinedvariables/data which can be acquired, generated and/or recorded in thecase of a scene shot S, of a view direction image M and/or in relationto a reference R. User inputs can also be received for particular steps,depending on which such criteria can be specified.

For example, following the quality class assignment GK in step S16, ananalysis software program of the device 22 can display as the firstaction the quality classes GK with their names and, as a text, acorresponding suggestion of an action or operation for the user. Thesoftware accordingly shows the user as the first action A1 based on thequality classes GK which view direction images were successful, whichshould be reviewed, or which are missing. The user can then specify oneor more of the quality classes GK as a selection AW and, with the aid ofthe software, view the corresponding selection AW in a linear/non-linearsuccession, for example, on the basis of a predefined sequence RF;review the view direction images M manually; and, where applicable,locally adjust them to his expectation, ignore them or delete them onthe reference image R. Manual reviewing can, for example, be provided byvisualizing the image as a third action A3, by the viewpoint B in thecorresponding image S of a scene video being visually displayed togetherwith its imaged viewpoint B′ on the reference R.

Actions, such as the first action A1, can also be executed automaticallyfor certain quality classes GK. For example, it is possible to ignore ordelete from view images M content which is to be seen in the referenceimage R but which is hidden by occlusions or objects in the scene imageS.

In general, the first and/or second and/or third action for a viewdirection image or view images M or for a selection of view images M canrepresent at least one of the following:

-   -   a graphical representation, displayable in particular on a        display device of the device 22,        -   on a reference R,        -   on a scene shot S,        -   as a time series representation for the view direction            images M,        -   by a highlighting of time intervals (such as segment            representation) on a time axis and/or        -   by means of different types of representation on a time axis            in relation to their selection and/or their quality class            membership and/or other intermediate results/final results    -   taking into consideration in a result/exported data        record/statistic,    -   ignoring and/or filtering in a result/exported data        record/statistic,    -   the deletion/removal of the view direction image(s) M,    -   the acceptance as valid view direction image(s) M and the saving        of such an acceptance for the corresponding view direction        image(s) M,    -   the ignoring as view direction image(s) M to be ignored and the        saving of such an ignoring action for the corresponding view        direction image(s) M,    -   a weighting of data in a result/exported data record/statistic,        or    -   a generation and/or transmission of a control signal, which can        affect a controller of a process or of a system or of a device,        which is dependent on a view direction image M, or on results        which are based on a view direction image M.

In addition, the quality evaluation can also be used for a furtheranalysis of the view images M and their results. A further embodimentprovides, for example, for a quality graph to be displayed on a displaydevice and for the user to be able to preset a threshold value byclicking it on the display device, i.e., by user input, and thus for twoquality classes GK to be specified. These classes may include, forexample, the classes ‘good’ or ‘bad,’ wherein the threshold value isspecified correspondingly by the user. This can be set globally for theentirety of the shots S or also locally for one or more of the sceneshots S. In this way, classes for time intervals can also be definedlocally. Furthermore, by means of the quality evaluation of view images,a quality report can also be created, for example, based on the merging(on different levels and in many ways) of view direction images M and anoverview representation of values, including statistically, for example,also for the scene shots S of multiple participants in a study orrepeated scene shots of a single participant, and so on. In addition, apivotable utilization of results is made possible.

FIG. 5 shows a schematic representation of a device 22 for evaluating atleast one view image in accordance with an exemplary embodiment of theinvention. In this case, the device 22 has a control unit taking theform of an evaluation unit 24. Furthermore, the device 22 has a firstinterface 26, via which input data 28 can be supplied to the device.Such input data 28 include data concerning the scene shots S, thereference R, as well as the view data, such as the viewpoints B providedin relation to the scene shots S and the viewpoints B′ which may havebeen imaged correspondingly onto the reference R. The input data 28 mayfurthermore also optionally include metadata MD, for example, inrelation to the scene shots S, the reference R, the viewpoint data B, B′or in general to a specific test, experiment or study to which theseinput data 28 are provided. Now, the evaluation unit 24 is designed toevaluate the respective view images M, which image the viewpoints Bdefined in relation to a respective scene shot S onto the reference R,in accordance with at least one specified quality measurement GM and toassign them to a quality class GK depending on this evaluation.Furthermore, the device 22 may also include a second interface 30 forreceiving a user input 32. Depending on the user input 32, for example,one or more of the predefined quality measurements GM can be selected inaccordance with which the respective view images are evaluated. Inaddition, the device 22 may also be designed to perform all the furthersteps described in connection with FIG. 4, such as making a selectionAW, defining a sequence RF and/or performing first, second and/or thirdactions A1, A2, A3. The device may also have a display device (notshown), for example, in order to display results E1, E2, E3 or to outputrecommendations or information to the user. In this way, on the basis ofthe quality class GK assigned to a view direction image M, an action canadvantageously be applied directly to the view direction image M as afunction of the quality class GK. A selection AW can also be created fora particular quality class GK, and an action can in turn be applicableto the selection AW. The selection AW can in this case be present in achronologically ordered sequence RF or even in a sequence RF determineddifferently by the device 22. Full quality evaluations can thus becarried out by an efficient method implemented by the device 22, saidmethod deciding about actions directly and/or in real-time and thusleading to the direct and/or real-time further processing of viewdirection images M.

Accordingly, the device 22 is also designed to, on the basis of thequality class assignment, make a selection AW of view direction imagesM, to which actions can be applied with no time lag. The device 22 canalso specify a sequence of the view direction images M within aselection AW and/or between selections AW, to which one or more actionscan then be applied in this thus specified sequence RF. The device 22[can] decide correspondingly about an action to be performed on aselection AW or leave the decision to a user. After the quality classassignment of the view direction images M has been done, a user can makea selection AW by specifying himself the quality classes GK relevant tothe selection AW (the set of quality classes used for the selection). Auser can furthermore modify a selection AW made by the device 22 or madeby himself, by manually adding and/or removing one or more viewdirection images M and/or manually changing the set and selection AW ofquality classes GK used for the selection AW of view direction images M.Via the user interface 30, the user can also specify the internal andexternal sequence RF of view direction images M.

The device 22 is accordingly designed to make a decision about a qualityclass assignment; an action which is applied to a view direction imageM; a selection AW of view direction images M; an (internal) sequence RFwithin a selection AW of view direction images M by sorting the viewdirection images M of the selection AW; an (external) sequence RFbetween selections AW of view direction images M; and/or an action whichis applied to a selection AW of view direction images M; making saiddecision in conjunction with the application of one or more rules and/orone or more classification methods with or without a model. Thedecisions can also be arrived at by the combination of thenon-exhaustively mentioned rules, classification methods with andwithout model. For example, sequences RF of selections AW or withinselections AW can also be created via properties of the scene shot S, ona chronologically linear basis or according to the influence on a resultstatistic relating to the imaged view directions/view endpoints B′. Aninternal and external sequence RF can be created by using sortingalgorithms. The datum/data used for this purpose can be any inputresults and/or previously determined intermediate results. In addition,the quality review can be carried out as a function of the objective ofthe review, which can, for example, be:

-   -   Impression gained of the overall quality of the statistics:        Summary of the overall quality through statistically        representative, exemplary sections of all view direction images;    -   Improvement of the overall quality by correcting the selected        view direction images (including automatic responses to        events/annotations, compliance with statistical targets for        overall quality and correction time);    -   Optimization of effort expended on the review/correction in        regard to the impact on the statistics (margins of the AOIs,        variation analysis);

The invention and its embodiments thus as a whole enable the definitionof quality evaluations of view images and offer numerous possibilitiesfor utilizing such quality evaluations.

LIST OF REFERENCE SYMBOLS

10 Person

12 Scene

14 Glasses

16 Scene camera

18 Bottle

20 s First predefined image area

20 r Second predefined image area

22 Device

24 Evaluation unit

26 Interface

28 Input data

30 User interface

32 User input

A1, A2, A3 Action

AW Selection

B Viewpoint

B′ Imaged viewpoint

E1, E2, E3 Result

GK Quality class

GM Quality measurement

GW Quality value

M View image

MD Metadata

R Reference

RD Reference data

RF Sequence

S Scene shot

1-20. (canceled)
 21. A method comprising: generating a view image basedon a plurality of scene images of a scene and a corresponding pluralityof eye tracking measurements; evaluating the view image according to atleast one predefined quality measurement; and providing a result of theevaluation of the view image.
 22. The method of claim 21, whereingenerating the view image includes: determining, based on the pluralityof scene images of the scene and the corresponding plurality of eyetracking measurements, a corresponding plurality of viewpoints of areference image of the scene.
 23. The method of claim 22, wherein atleast one of the plurality of scene images of the scene is from adifferent perspective than the reference image of the scene.
 24. Themethod of claim 22, wherein evaluating the view image includesdetermining a similarity between at least one of the plurality of sceneimages of the scene and the reference image of the scene.
 25. The methodof claim 22, wherein evaluating the view image includes determining asimilarity between a first area of at least one the plurality of sceneimages of scene and a second area of the reference image of the scene,wherein the first area is defined by the corresponding eye trackingmeasurement and the second area is defined by the correspondingviewpoint.
 26. The method of claim 22, wherein evaluating the view imageincludes determining a distribution of the plurality of viewpoints. 27.The method of claim 22, wherein providing a result of the evaluation ofthe view image includes displaying a visualization of the plurality ofviewpoints with respect to the reference image of the scene in whichquality assigned to respective ones of the plurality of viewpoints isidentified visually.
 28. The method of claim 27, wherein respective onesof the plurality of viewpoints are displayed in different colorsaccording to the quality assigned to the respective viewpoint.
 29. Themethod of claim 21, further comprising: generating a second view imagebased on a second plurality of scene images of the scene and acorresponding second plurality of eye tracking measurements; evaluatingthe second view image according to the at least one predefined qualitymeasurements; and comparing the result of the evaluation of the secondview image with the result of the evaluation of the view image.
 30. Themethod of claim 21, further comprising assigning the view image one of aplurality of quality classes based on the result of the evaluation. 31.An apparatus comprising: a scene camera to capture a plurality of sceneimages of a scene; an eye tracker to generate a corresponding pluralityof eye tracking measurements; and a processor to: generate a view imagebased on a plurality of scene images of a scene and a correspondingplurality of eye tracking measurements; evaluate the view imageaccording to at least one predefined quality measurement; and provide aresult of the evaluation of the view image.
 32. The apparatus of claim31, wherein the processor is to generate the view image by determining,based on the plurality of scene images of the scene and thecorresponding plurality of eye tracking measurements, a correspondingplurality of viewpoints of a reference image of the scene.
 33. Theapparatus of claim 32, wherein the processor is to evaluate the viewimage by determining a similarity between at least a portion of one theplurality of scene images of the scene and at least a portion of thereference image of the scene.
 34. The apparatus of claim 32, wherein theprocessor is to evaluate the view image by determining a distribution ofthe plurality of viewpoints.
 35. The apparatus of claim 32, wherein theprocessor is to provide a result of the evaluation of the view image bydisplaying a visualization of the plurality of viewpoints with respectto the reference image of the scene in which quality assigned torespective ones of the plurality of viewpoints is identified visually.36. The apparatus of claim 31, wherein the processor is further toassign the view image one of a plurality of quality classes based on theresult of the evaluation.
 37. A non-transitory computer-readable mediumencoding instructions which, when executed, cause a processor to performoperations comprising: generating a view image based on a plurality ofscene images of a scene and a corresponding plurality of eye trackingmeasurements; evaluating the view image according to at least onepredefined quality measurement; and providing a result of the evaluationof the view image.
 38. The non-transitory computer-readable medium ofclaim 37, wherein the instructions, when executed, cause the processorto generate the view image by determining, based on the plurality ofscene images of the scene and the corresponding plurality of eyetracking measurements, a corresponding plurality of viewpoints of areference image of the scene.
 39. The non-transitory computer-readablemedium of claim 38, wherein the instructions, when executed, cause theprocessor to evaluate the view image by determining a similarity betweenat least a portion of one the plurality of scene images of the scene andat least a portion of the reference image of the scene.
 40. Thenon-transitory computer-readable medium of claim 38, wherein theinstructions, when executed, cause the processor to evaluate the viewimage by determining a distribution of the plurality of viewpoints.